Reliability and validity of Izard's Differential Emotions Scale

Izard's Differential Emotions Scale (DES) was administered to 204 University of Delaware undergraduates under each of four imaginal mood-induction conditions (labelled: General Depression, Curiosity, Specific Depression and Anxiety) and an actual pre-exam condition. A factor analysis of DES items supported the construct validity of some subscales. A repeated-measures multiple discriminant function analysis indicated that two-thirds of the DES items discriminated between the five conditions-an extremely stringent test of item validity. It was suggested, however, that refinement of the DES should attempt to decrease item redundancy (internal consistency) by reestablishing the instrument on a broader base of adjectives descriptive of fundamental emotions. Introduction Izard (1971, 1972, 1977, 1978, 1979a, b, 1980; Izard and Buechler, 1979, 1980) has postulated 10 fundamental emotions, universally discernible in the human facial expression. These fundamental emotions, which involve complex neuromuscular feedback loops via the trigeminal nerve (facial feedback hypothesis-see Zuckerman, Klorman, Larrance and Spiegel (1981)), have been operationalized in terms of the Differential Emotions Scale (DES; a 30 item self-report inventory, with each item scored on a 5-point Likert system)-see Izard, Dougherty, Bloxom and Kotsch (1974). The DES is purported to measure the subjective-experience component of the fundamental emotions. According to Izard et al. (1974, p. 1), "the DES focuses on ten discrete emotions, each of which was defined theoretically as having neural, neuromuscular-expressive, and experiential components. The components are interactive, but in the fundamental emotion process the common order of occurrence is: (a) efferent neural activity in the facial nerve and consequent patterned facial expression, (b) sensory feedback from the expression via the trigeminal nerve, (c) cortical integration of the sensory data and consequent subjective experience." Izard et al. contended that the DES reliably and validly divides the individual's description of his/her emotional experience into discrete categories of fundamental emotions. Unlike other multivariate measures of mood states, the DES is based on the assumption that mood states (such as anxiety or depressed mood) involve a characteristic pattern of fundamental emotions-a unique conceptualization. The DES has been subjected to numerous factor analyses on large samples, using a principal components plus Promax rotation methodology (e.g. Izard, 1972). Most of these factorings have supported at least eight of the postulated fundamental emotions. Difficulties were however apparent for the subscales of Anger, Disgust and Contempt, which tended to load on a common factor. The subscales of Interest, Joy, Surprise, Sadness, Fear, Shyness and Guilt, on the other hand, seem fairly well validated from the several factorings, although in one large study of 1182 Ss, the subscales of Fear and Sadness were not clearly separated (see Izard, 1972). More recently, Erode (1980) administered the DES to 230 mothers and after factoring the data reported the emergence of eight distinct factors, including those pertaining to Joy, Fear, Surprise, Interest, Shyness, Contempt and Guilt. The subscales of Anger and Sadness were combined as a single factor. Moreover, no distinct factor emerged for the subscale of Disgust and, also, 2 of the 30 DES items failed to load on any of the empirically-derived factors (Emde, 1980, p. 24). Fuenzalida, Emde, Pannabecker and Stenberg (1981) subjected the DES item responses of 613 mothers to an iterative principal-components analysis, and rotated to both Varimax and Oblimax criteria [factor extraction number was determined by the old eigenvalue > 1 rule-an unsatisfactory procedure generally-see Barrett and Kline (1982)]. The orthogonal solution was retained (despite its being only a special instance of the more appropriate oblique solution-see Loo (1979)]. Nevertheless, eight of the DES subscales received factorial support (namely: Anger, Surprise, Interest, Contempt, Fear, Guilt and Shyness), including a mixed factor relating to Joy and Sadness. Five of the DES items failed to load on any factor (see Fuenzalida et al., 1981, p. 40). Fuenzalida et al. therefore asserted that the DES was sufficiently construct valid to justify its continued use in applied studies. Yet, Mosher and White (1981, p. 62) claimed only that six to eight of the postulated fundamental emotions has been confirmed empirically. The present study sought to provide further evidence on this issue, and on the validity of the items. Subjects Method The sample comprised 204 University of Delaware undergraduates (approx. 90"/. Were females) whose mean age was 20.46 years (SD = 2.78 yrs). Students ranged in age from 18 to 45 years. Participation was voluntary, and those who participated appeared to take the study seriously and to follow instructions without reservation. Design and procedure The study involved four imaginal treatments (General Depression, Curiosity, Specific Depression and Anxiety), as well as an actual pre-examination condition. Izard (1972), Schwartz and Weinberger (1980), as well as Sirota and Schwartz (1982), have all demonstrated that imaginal mood treatments effectively induce moods such as anxiety or depression (a well-known phenomenon in the behaviour modification technique of systematic desensitization, for example). In the General Depression condition, students responded to the DES items as though they felt 'depressed'. In the Specific Depression condition, they imagined that they were about to take an exam which they felt they would fail, making them feel depressed. Presentation of the four imaginal treatments was counterbalanced to avoid position effects. Use of the preexamination condition (on a separate occasion) allowed some assessment of external validity (cf. Campbell and Stanley, 1966; Cook and Campbell, 1979). Results and Discussion Under the five conditions, the DES subscales of Joy, Surprise, Contempt, Fear and Shame/Shyness exhibited the highest lower-bound estimates of test-retest reliability with correlations up to about 0.7, which for state measures given under different conditions is good (cf. Boyle, 1979, p. 78). However, neither the Interest subscale nor the Guilt subscale demonstrated significant retest correlations in a comparison of the General and Specific Depression conditions, and also of the Specific Depression and pre-examination conditions. Izard et al. (1974, p. 32) reported a retest correlation of 0.77 for the trait version of the DES (using a 5-point frequency rather than intensity response system), which seems somewhat low for a reliable self-report instrument of the trait variety (cf. Boyle, 1979, p. 78). According to Izard et al. (1974, p. 32), the reliability of the state version of the DES can only be assessed in terms of internal consistency of the items in each subscale. However with only three items per subscale which may allow rapid measurement of transient moods although increased items in accord with the Spearman-Brown formula should result in greater reliability of the DES subscales-see Nunnally (1978, pp. 210212, 243-245) one would expect high internal consistency if the items were measuring essentially the same limited aspect of the particular dimension. What is more desirable are items which have less homogeneity of variance (less internal consistency and item redundancy), but which correlate highly with the relevant factor. This would enable each item to measure a different aspect of a given dimension, providing broader, less redundant and more efficient measurement. As Kline (1979, p. 3) indicated, α coefficients should not exceed 0.7 if each item is to add something new to the measurement of a construct. Likewise, Cattell (1982) has discussed this issue, along with Allen and Potkay (1983, p. 1088) and Lachar and Wirt (1981), who have all indicated that internal consistency is not particularly related to reliability. Izard et al. (1974) reported a mean α coefficient for the DES subscales of 0.81. In the present study, the DES total scale exhibited α coefficients of 0.78, 0.86, 0.86, 0.84 and 0.89, respectively, for the five treatment conditions. For the 10 subscales, α estimates ranged from 0.56 to 0.88, which suggests some item redundancy. To check the construct validity of the 10 DES subscales, an iterative principal factoring was performed on the item intercorrelations. Eight factors on the basis of the Scree test (cf. Barrett and Kline, 1982) were rotated to oblique simple structure for the General Depression condition using a direct oblimin procedure (see Nie, Hadlai Hull, Jenkins, Steinbrenner and Bent, 1975). The ±0.10 hyperplane count (Cattell, 1978, p. 142) was 60.42%. While a seven-factor oblique solution accounted for 57.62% of variables in the hyperplane, a nine-factor solution produced a hyperplane count of 60.41% which was no better than that for the eight-factor solution. Interestingly, had the eigenvalues >1 factor extraction rule been applied, only seven factors would have been retained for subsequent rotation, and one factor would have been lost. Examination of the oblique factor pattern reveals that the four DES subscales of Contempt, Surprise, Fear and Disgust were clearly defined by Factors 3, 5, 6 and 7, respectively. Nevertheless, Item 23, "Feel like you are better than somebody", failed to load significantly on Factor 3. For Factor 6, Item 7, "Feel shy, like you want to hide", also loaded significantly. For Factor 7, Item 22, "Feel like things are so rotten they could make you sick", failed to have a significant loading, while Item 1, "Feel regret, sorry about something you did", loaded significantly on this factor. Factor 4 represented a combination of the Sadness and Anger subscales which suggests that the instructions to students may have simultaneously induced some hostility, presumably due to the inconvenience of having to experience a negative mood state (depression), when no such state already existed. Factor 1 represented a combination of the Guilt, Sadness and Shame/Shyness subscales, while Factor 2 represented a grouping of the Joy, Surprise and Interest subscales. Hence Factor I involved a cluster of negative mood states, whereas Factor 2 involved a cluster of positive moods reminiscent of Eysenck's (e.g. Lynn, 1981) personality dimensions of Introversion and Extraversion. Factor 8 seemed to represent a state of depressed mood indicative of the condition under which this data was obtained. These factor analytic findings, therefore, provided only partial support for the construct validity of Izard's DES subscales. Since some factors involved combinations of the postulated fundamental emotions, it appears that the division of the DES into 10 subscales may be partly an artefact of the factor-analytic procedures employed in Izard's studies (cf. Izard, 1972). Given the clustering of these subsca1es, it was deemed necessary to examine the discriminant validities of the DES items, using a repeated-measures application of multiple discriminant analysis (after Nunnally, 1981). The 30 DES items were tested for their ability to individually discriminate between the five treatment conditions-an extremely stringent test of the sensitivity of the items. Twenty of the 30 items discriminated significantly. Had only two or three conditions been employed, several of the remaining 10 items probably also would have been significant discriminators. F-tests on the power of the discriminant functions were all highly significant (p < 0.001). The first discriminant function accounted for most of the variance (67.41%) and therefore was considered as the most important for examining the discriminant validities of the DES items. Table 2 presents the data pertaining to the four discriminant functions. Examination of the correlations of the 20 significant DES items with the first discriminant function (see Table 3), revealed that Item 23 (from the Contempt subscale) correlated only 0.16 with the function. While this was statistically significant (p < 0.05), compared to the other correlations it was low. Therefore of the 30 DES items, a stringently conservative estimate is that 19 items appear to be highly valid discriminators between various mood states. The stepwise discriminant analysis sequentially selected those DES items which contained most of the classificatory information (cf. Amick and Walberg, 1975, pp. 244-251). All three items for the subscales of Interest, Surprise and Sadness, were significant discriminators between the five conditions. For the subscales of Joy, Anger, Fear and Shame/Shyness, two items were significant discriminators, while for the Disgust, Contempt and Guilt subscales only one item discriminated significantly between the five conditions. The above findings suggest that while at least two-thirds of the DES items are sensitive indicators of different mood states, the construct validity of the DES subscales is not clear. Only for the subscales of Contempt, Surprise, Fear and Disgust, was a Table 1 Oblique factor pattern solution Item Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 Factor 8 I 0.30 -0.10 -0.14 0.05 0.06 0.06 0.52 -0.08 2 0.03 0.75 -0.07 -0.05 0.06 0.07 0.12 0.10 3 -0.13 0.08 0.19 0.09 -0.06 -0.12 0.60 -0.01 4 0.37 -0.37 -O.03 0.33 0.11 -0.05 0.07 0.02 s -0.02 0.11 -O.02 0.05 -0.49 -0.01 0.04 -0.06 6 0.10 -0.03 0.72 0.12 -0.02 0.00 0.02 0.06 7 0.37 -0.08 0.14 -0.15 0.06 -0.37 -0.04 -0.07 8 0.02 0.67 -0.13 -0.07 -0.04 -0.04 0.03 0.00 9 0.01 -0.01 -0.06 0.09 -0.03 -0.89 0.07 0.09 10 -0.10 0.03 0.09 0.63 -0.06 -0.13 0.12 0.11 11 -0.02 0.80 -0.02 -0.17 -0.03 0.02 0.09 0.06 12 -0.03 -0.04 0.82 -0.02 0.07 0.03 0.06 -0.01 13 0.00 0.60 -0.26 0.04 -0.08 -0.02 0.00 0.16 14 0.09 -0.10 -0.02 0.01 -0.66 0.03 -0.01 0.08 IS -0.02 -0.09 0.01 0.04 0.02 -0.68 0.04 -0.02 16 -0.08 0.04 0.00 0.72 0.05 -0.07 0.07 -0.07 17 0.33 -0.39 0.04 0.34 O.Dl -0.05 O.Q3 -0.01 18 0.35 -0.07 -0.02 0.17 0.02 -0.03 0.22 -0.35 19 0.64 0.24 0.09 -0.08 -0.17 -0.27 0.11 -0.08 20 -0.02 -0.19 0.22 0.11 -0.20 -0.16 0.32 -0.14 21 -0.04 0.66 0.03 -0.20 -0.13 -0.06 -0.14 -0.08 22 -0.02 -0.18 0.21 0.20 0.14 -0.18 0.07 -0.39 23 0.01 0.48 0.18 0.14 0.01 0.10 -0.13 0.00 24 0.13 -0.15 -0.03 0.00 0.05 0.01 0.14 -0.63 25 -0.16 0.11 0.04 -0.08 -0.41 -0.05 -0.03 -0.42 26 0.00 0.78 -0.04 0.06 -0.08 0.05 -0.07 -0.09 27 0.14 -0.12 0.16 0.61 -0.04 0.03 -0.04 0.06 28 0.24 -0.21 0.09 0.43 0.()7 -0.07 0.04 -0.14 29 0.38 -0.04 0.17 0.15 -0.02 -0.20 0.05 -0.23 30 0.16 0.06 -0.08 0.20 0.08 -0.45 -0.10 -0.40 Percentage of variance ±0.10 Hyperplane 31.3 19.2 13.2 10.3 8.6 7.2 6.3 4 count 17 13 18 14 23 20 20 20 Significant loadings are underlined. Scoring key for DES is as follows Emotion subscale Items Interest 8 13 26 Joy 2 II 21 Surprise 5 14 25 Sadness 4 17 28 Anger 10 16 27 Disgust 3 20 22 Contempt 6 12 23 Fear 9 15 30 Shame/Shyness 7 19 29 Guilt I 18 24 Table 3. Correlations of the DES items with first discriminant function (N = 204) Item r Item I -0.66 IS -0.41 2 0.60 17 -0.72 Table 2. Canonical discriminant functions for the DES items (N = 204) 3 -0.43 21 0.65 4 -0.69 23 0.16 Percentage Canonical 5 0.58 25 0.58 Function Eigenvalue of variance correlation 7 -0.45 26 0.82 1 1.22 67.41 0.742 8 0.77 27 -0.69 2 0.36 19.57 0.512 10 -0.52 28 -0.73 3 0.19 10.64 0.402 13 0.81 29 -0.49 4 0.04 2.38 0.204 14 0.55 30 -0.46 clearcut matching of the subscale with the empirical factor achieved in the present study. While there is an apparent need to improve the construct validity of the DES subscales (perhaps different subscales are required), for reducing item redundancy (internal consistency), for improving the retest reliability of the instrument, Izard must be commended for his attempt to measure fundamental emotions. His differential emotions theory represents an advance in conceptualization, and his DES follows logically from his theory. Unfortunately there are many problems with self-report measures, such as the transparency of items, which therefore lend themselves to distortion, ranging all the way from inadequate self-insight to deliberate faking. What is needed in this area are measures which comprise objective items, which have no readily discernible connection with the moods being measured (content validity is no guarantee of validity). In the absence of objective instruments, the best that can be achieved is to utilize scales such as the DES (with its unique conceptualization of patterns of fundamental emotions), and to further refine the various subscales and items in order to improve its reliability and validity. Perhaps also, evidence in support of additional fundamental emotions might appear in a clinical sample in which confirmatory factor analysis is applied. It would seem useful to look at target groups identified as in crisis situations (similar to recent critical item subsets for the MMPI), in the search for additional fundamental emotions. In brief, the DES potentially has much to offer, provided that it is adequately refined and developed. Acknowledgements The author is indebted to Dr Cal Izard for his kind advice and to Dr Jum Nunnally (who regrettably died on 22 August 1981) for his kind and indispensableadvice. Dr Nunnally suggested the repeated-measures application of the multiplediscrminant function analysis on the DES items. ReferencesAllen B. P. and Potkay C. R. (1981) On the arbitrary distinction between states andtraits. J. Person. soc. Psychol. 41, 916-928.Allen B. P. and Potkay C. R. (1983) Just as arbitrary as ever: comments on Zuckerman'srejoinder. J. Person. soc. Psychol. 44, 1087-1089.Amick D. J. and Walberg H. J. (1975) Introductory Multivariate Analysis: forEducational, Psychological, and Social Research. McCutchan, Berkeley, Calif.Barrett P. T. and Kline P. (1982) Factor extraction: an examination of three methods.Person. Study Gp Behav. 2, 84-98. Boyle G. J. (1979) Delimitation of state-traitcuriosity in relation to state anxiety and learning task performance. Aust. J. Educ.23, 70-82.Campbell D. T. and Stanley J. C. (1966) Experimental and Quasi-experimental Designsfor Research. Rand McNally, Chicago, Ill.Cattell R. B. (1978) The Scientific Use of Factor Analysis in Behavioral and LifeSciences. Plenum Press, New York.Cattell R. B. (1982) The psychometry of objective motivation measurement: a response to the critique of Cooper and Kline. Br. J. educ. Psycho/. 52, 234-241. Cook D. and Campbell D. T. (1979) Experimental and Quasi-experimentation: DesignAnalysis Issues for Field Settings. Rand McNally, Chicago, Ill.Emde R. N. (1980) Levels of meaning for infant emotions. In The Minnesota Symposia on Child Psychology; Vol. [3, Development of Cognition, Affect, and Social Relations (Edited by Collins W. A.). Erlbaum, Hillsdale, N.J.Eysenck H. J. and Eysenck S. B. G. (1976) Personality Structure and Measurement.Routledge & Kegan Paul, London. Fuenzalida C., Emde R. N., Pannabecker B. J. and Stenberg C. (1981) Validation of the Differential Emotions Scale in 613mothers. Motivn. Emotion 5, 37-45. Izard C. E. (1971) The Face of Emotion. Appleton-Century-Crofts, New York.Izard C. E. (1972) Patterns of Emotions. Academic Press, New York. Izard C. E. (1977)Human Emotions. Plenum Press, New York.Izard C. E. (1978) On the development of emotions and emotion-cognition relationshipsin infancy. In The Development of Affect (Edited by Lewis M. and RosenblumL.A.). Plenum Press, New York.Izard C. E. (1979a) Emotions as motivations: an evolutionary-developmentalperspective. In Nebraska Symposium on Motivation (Edited by Dienstbier R.A.). Univ. of Nebraska Press, Lincoln, Neb.Izard C. E. (1979b) Facial expression, emotion, and motivation. In Nonverbal Behavior(Edited by Wolfgang A.). Academic Press, New York.Izard C. E. (1980) The emergence of emotions and the development of consciousness ininfancy. In The Psychobiology of Consciousness (Edited by Davidson J. M. andDavidson R. J.). Plenum Press, New York.Izard C. E. (Ed.) and Buechler S. (1979) Emotion expressions and personalityintegration in infancy. In Emotions in Personality and Psychopathology. Plenum